Search results for "File systems"

showing 3 items of 3 documents

GekkoFS - A Temporary Distributed File System for HPC Applications

2018

We present GekkoFS, a temporary, highly-scalable burst buffer file system which has been specifically optimized for new access patterns of data-intensive High-Performance Computing (HPC) applications. The file system provides relaxed POSIX semantics, only offering features which are actually required by most (not all) applications. It is able to provide scalable I/O performance and reaches millions of metadata operations already for a small number of nodes, significantly outperforming the capabilities of general-purpose parallel file systems. The work has been funded by the German Research Foundation (DFG) through the ADA-FS project as part of the Priority Programme 1648. It is also support…

File system020203 distributed computingBurst buffersParallel processing (Electronic computers)Computer scienceProcessament en paral·lel (Ordinadors)020207 software engineering02 engineering and technologyBuffer storage (Computer science)computer.software_genreData structureDistributed file systemsMetadataParallel processing (DSP implementation)POSIXServerScalabilityHPC0202 electrical engineering electronic engineering information engineeringOperating systemHigh performance computingDistributed File System:Informàtica::Arquitectura de computadors::Arquitectures paral·leles [Àrees temàtiques de la UPC]computerCàlcul intensiu (Informàtica)2018 IEEE International Conference on Cluster Computing (CLUSTER)
researchProduct

File system scalability with highly decentralized metadata on independent storage devices

2016

This paper discusses using hard drives that integrate a key-value interface and network access in the actual drive hardware (Kinetic storage platform) to supply file system functionality in a large scale environment. Taking advantage of higher-level functionality to handle metadata on the drives themselves, a serverless system architecture is proposed. Skipping path component traversal during the lookup operation is the key technique discussed in this paper to avoid performance degradation with highly decentralized metadata. Scalability implications are reviewed based on a fuse file system implementation. Peer Reviewed

Information storage and retrieval systemsComputer scienceDistributed computingInterface (computing)Key-value storages02 engineering and technologycomputer.software_genreObject storagesLookupsComputer clusterServerData_FILES0202 electrical engineering electronic engineering information engineering:Informàtica::Arquitectura de computadors [Àrees temàtiques de la UPC]Virtual storageFile systemMetadataFile organizationScalability020206 networking & telecommunications020207 software engineeringFile systemsObject storageMetadataKineticsInformació -- Sistemes d'emmagatzematge i recuperacióScalabilityGrid computingSystems architecturecomputerCluster computing
researchProduct

GekkoFS — A Temporary Burst Buffer File System for HPC Applications

2020

Many scientific fields increasingly use high-performance computing (HPC) to process and analyze massive amounts of experimental data while storage systems in today’s HPC environments have to cope with new access patterns. These patterns include many metadata operations, small I/O requests, or randomized file I/O, while general-purpose parallel file systems have been optimized for sequential shared access to large files. Burst buffer file systems create a separate file system that applications can use to store temporary data. They aggregate node-local storage available within the compute nodes or use dedicated SSD clusters and offer a peak bandwidth higher than that of the backend parallel f…

Information storage and retrieval systemsPOSIXFile systemBurst buffersComputer scienceProcess (computing)computer.software_genreDistributed file systemsComputer Science ApplicationsTheoretical Computer ScienceMetadataInformació -- Sistemes d'emmagatzematge i recuperacióComputational Theory and MathematicsHardware and ArchitecturePOSIXHPC:Informàtica::Sistemes d'informació::Emmagatzematge i recuperació de la informació [Àrees temàtiques de la UPC]ScalabilityOperating systemBandwidth (computing)High performance computingIsolation (database systems)Càlcul intensiu (Informàtica)computerSoftwareJournal of Computer Science and Technology
researchProduct